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0. Introduction. 

Let O C C"^ be a bounded, pseudoconvex domain with C°° boundary, equipped with 
the standard Hermitian metric inherited from C"^. The 9-Neumann problem for (p, 
forms in Q is the boundary value problem 



□w = 


f 


in Q 


u_idp = 





on dn 


Bujdp = 





on dfl 



where p is a defining function for O, □ = dd* + d*d, u, f are [p, q) forms, and J denotes 
the interior product of forms. Under the stated hypotheses on Q, this problem is uniquely 
solvable for every / G L^(0). The Neumann operator A^, mapping / to the solution 
u, is continuous on L^(0). The Bergman projection B is the orthogonal projection of 
L^(0) onto the closed subspace of holomorphic functions on O, and is related to by 
B = I- d*Nd. 

N and B are C°° pseudolocal if O is strictly pseudoconvex, or more generally, is of 
finite type [Cal]. Both preserve C°°(0) under certain weaker hypotheses [BS2],[Ca2]. For 
any pseudoconvex, smoothly bounded O and any finite exponent s, there exists a strictly 
positive weight w G C°°(0) such that the Neumann operator and Bergman projection 
with respect to the Hilbert space L'^{Q,w{x)dx) map the Sobolev space H^{Q) boundedly 
to H\n), for aU < t < s [Kl]. It has remained an open question whether and S, 
defined with respect to the standard metric, preserve C°°(0) without further hypotheses 
on O. An affirmative answer would have significant consequences [BL]. 
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Theorem. There exist pseudoconvex, smoothly bounded domains Q for which the 

Neumann operator on (0, 1) forms and Bergman projection fail to preserve C°°{Q). 

Examples are the worm domains, originally introduced by Diederich and Fornaess [DF] 
for another purpose-*^ but long considered likely candidates for N and B to fail to be globally 

regular in C°°. 

The proof depends on the observation of Barrett [B] that for each worm domain W, for 
all sufficiently large s, N and B fail to map H^{W) boundedly to H^{W). We establish 
for each worm domain an a priori inequality of the form ||A^/||h^ < C'sH/Hi/^, valid for all 
/ e C°°{W) such that Nf e C°°(>V), for a sequence of exponents s tending to oo. UN 
were to preserve C°°(VV), then since it is a bounded linear operator on L^(>V) and since 
(7°°(>V) is dense in H^{yV), it would follow that N maps H^{W) boundedly to itself, for 
a sequence of values of s tending to oo, a contradiction. More accurately, our inequality is 
valid only for certain subspaces of ( W) preserved by □, but this still suffices to contradict 
the theorem of Barrett. 

An analogous counterexample in the real analytic context is already known [Chi] : there 
exists a bounded, pseudoconvex domain O C having real analytic boundary, such that 
the Szcgo projection fails to preserve C"^(9ri). That result and its proof are however not 
closely related to the C°° case. 

§§1 through 3 review material on worm domains and the 5-Neumann problem, and 
present some routine but tedious reductions. §4 formalizes a class of two-dimensional 
problems subsuming those to which the reductions lead. The analysis of those problems is 
contained in §§5 and 6. 

1. Reduction to the Boundary. 

The 5-Neumann problem is a boundary value problem for an elliptic partial differential 
equation, and as such is amenable to treatment by the method of reduction to a pseu- 
do differential equation on the boundary. This reduction has been carried out in detail 
for domains in by Chang, Nagel and Stein [CNS]. We review here certain of their 
computations and direct consequences thereof. 

Assume fl C to be a smoothly bounded domain. The equation Dtt = f on Q for 
(0, 1) forms in C°°(0) is equivalent to an equation D'^v = g on dfl, where v, g are sections 
of a certain complex line bundle^ B^'^. Let p be a smooth defining function for Q and 
define 0)2 = dp, and uji = {dp/dz2)dzi — {dp/dzi)dz2- 

V is related to u hy u = Pv + Gf, where P, G are respectively Poisson and Green 
operators for the elliptic system = / with Dirichlet boundary conditions. In particular, 
if / e C°°(ri), then u e C°°{^1) if and only if the same holds for v. More precisely, G maps 



^Some but not all worm domains have nontrivial Nebenhulle[FS, p. Ill], whereas all worm domains 
are counterexamples to global regularity. 

'^B^'^ is defined to be the quotient of the restriction to dfl of T^'^C'^, modulo the span of dp. Sections 
of may be identified with scalar-valued functions times coi, hence with scalar-valued functions. 
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H%n) to while P maps H^{dn) to H^+^/'^{n), for each s>0. Thus ii f e H\ 

in order to conclude that w G i?* it suffices to know that v E 

On the other hand, g = {dGf jdp), restricted to dQ. If f e then dGf G H'+\n), 
so its restriction to the boundary belongs to i?^+^/^(9ri). Thus in order to show that 
N preserves H^{n) it suffices to show that if D+v G H'+'^/^{dn), then v G H'-^/'^{dn), 
assuming always that s > 1/2. 

On dn a Cauchy-Riemann operator is the complex vector field L — {dz^ p) — ((^Z2 P) • 
Define L to be the complex conjugate of L. The characteristic variety"^ of L is a real line 
bundle V. Assuming O to be pseudoconvex and the set of strictly pseudoconvex points to 
be dense in dO,, T splits smoothly and uniquely as r+ U F", where each fiber of r='= is a 
single ray, and where F"'" is distinguished from F~ by the requirement that the principal 
symbol of \L^L\ is nonpositivc on r^, modulo terms spanned by the symbols of the real 
and imaginary parts of L and a term of order 0. Equivalently, the principal symbol of 
[L, L*] is nonnegative on r+, modulo the same kinds of error terms. 

□+ is a classical pseudodifferential operator of order +1. Its principal symbol vanishes 
everywhere on F""" but nowhere else. Microlocally in a conic neighborhood of F^, □+ takes 
the form 

□+ = QLL + FiL + F2L + F3 

where Q is an elliptic pseudodifferential operator of order —1, and each Fj is a pseudodif- 
ferential operator of order less than or equal to —1. Since is elliptic except on F"*", for 
any pseudodifferential operator G of order zero whose symbol vanishes identically in some 
neighborhood of F+ , one has for all u G C°° and sl\ N < 00 

(1-1) \\Gu\\Ht+i(dii) < CWW^uWutidO) + CN\\u\\H-N{^^a.)■ 

'Lel A be an elliptic pseudodifferential operator of order +1 such that Ao Q equals the 
identity on LP'^dVt), modulo an operator smoothing of infinite order. Composing on the 
left with A, the equation D+v = g may be rewritten os, 2,v = g microlocally in a conic 
neighborhood of F+, where 

(1.2) SI = LL + BiL + B2L + B3, 

ll^ll/f* < C'||(7||//t+i + CatIIi'II j;/-jv for any finite and each Bj is an operator of order 
less than or equal to zero. Therefore in order to show that the Neumann operator satisfies 
an a priori inequality of the form \\Nf\\H^{n) < C\\f\\H^{n) for all / G C°^{fl) such that 
Nf G C°°(f2), it suffices to establish an priori inequality for all v G C°°{dCl) of the form 

(1.3) \\v\\Ht < C\\Slv\\Ht + C\\v\\Ht' + C\\Qv\\h^+2 

where t = s — 1/2, for some t' < t and some pseudodifferential operator Q of order zero 
whose symbol vanishes identically in some neighborhood of F"*". 

^By the characteristic variety of a pseudodifferential operator we mean the conic subset of the cotangent 
bundle on which its principal symbol vanishes. 
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2. Worm Domains. 

A worm domain in is an open set of the form 

W = {z: 1^1 +e^^°sN2|^|2 ^ ^ _ ^^^Qg |^^|2^| 

where the function (p vanishes identically on some interval [— r, r] of positive length, and 
is constructed [DF] so as to guarantee that W will be pseudoconvex with boundary, 
and will be strictly pseudoconvex at every boundary point except those on the exceptional 
annulus A C dW defined as 

A = {z : zi = and | log \z2\'^\ < r}. 

The circle group acts as a group of automorphisms of W by z ^-> Rqz = (^i, 6*^22) • 
It acts on functions by Refi^z) = f{Rez), and on (0, 1) forms by Ro{fidzi + /2(i^2) = 
{Reh)dzi + {Rgf2)e-'^dz2. The Hilbert space ^(o,fe)(^) of square integrable (0, k) forms 
decomposes as the orthogonal direct sum ^j^z'Hj where Hj is the set of all (0, k) forms 
/ satisfying Ref = e^^^ f . d is an unbounded linear operator from to Ti^"*"^, B maps 
to itself, and the Neumann operator maps ?ij to ?ij boundedly, for each j. 

For each k and each s > 0, the Sobolev space H^{W) likewise decomposes as an orthogo- 
nal direct sum of subspaces . It is known that for any smoothly bounded, pseudoconvex 
domain f2 C C^, for any exponent s > 0, if AT maps H^{fl) boundedly to itself, then B also 
maps H^{Q) boundedly to itself [BSl], where denotes in the first instance a space of 
one forms, and in the second, a space of functions. Because N, B preserve the summands 
Tij, the same proof shows^ that for any fixed j, if N maps the space Hj of (0, 1) forms 
boundedly to itself, then B maps the space Hj of functions boundedly to itself. Barrett 
[B] that for each worm domain, for all sufficiently large s, for all j, B fails to map i7| 
boundedly to itself. Therefore in order to prove that AT, acting on (0, 1) forms, fails to 
preserve C°°(>V), it suffices to establish the following result for a single index j. 

Proposition 1. For each worm domain there exists a discrete subset S C R"*" such that 
for each s ^ S and each j & Ti there exists Cgj < 00 such that for every (0, 1) form 
uen^n C°° (W) such that Nu e C°° (W), 

(2.1) \\Nu\\hs{^w) <Cs,j\\u\\H-{W)- 

The defining function p = 1 — 0(log \z2\'^) — \zi + e*^"^!^^!^^ for W is invariant under 
Re, as is the (0, 1) form 0)2 defined above. u)i satisfies RaOJi = exp(— ia)u;i for all ct, but 
it may also be made invariant by multiplying it by the function {zi^re^^) ^ e*^, which is 
smooth in a neighborhood of W. We work henceforth with this modified uji. 

^This follows from the argument of Boas and Straube [BSl] because all elements of their proof may be 
chosen to be invariant under the automorphisms Rq. 
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□+ commutes with Re for all 6. Indeed, D+v = dPvjdp [CNS]. □ commutes with 

hence so must P. d commutes with Rq, and the Hermitian metric on and dp are 
likewise i^g-invariant. Thus all ingredients in the above expression for □+ are invariant, 
hence so is itself. 

Identify square integrable sections of with scalar- valued functions as above, and 
decompose L'^{dW) = ®Hj{dW) where Hj is the subspace of those functions satisfying 
Rf)f = e'^^f. Then □+ maps HjidW) n to HjidW). We have seen in §1 that 
Proposition 1 would be a consequence of the validity of (1.3) for all v e C°°{dyV) n Tij, 
fort = s- 1/2. 

Fix j and assume henceforth that u belongs to 7f](>V) and to C°°. Then the associated 
boundary function v belongs to Hj fl C°°{dW). Henceforth we work exclusively on the 
boundary, and simplify notation by writing simply Tij rather than T-Cj{dW). 

Note that L, L take Hj fl to Hj-^i and to Hj-i, respectively. The operator A 
introduced after (1.1) may be constructed to be i^^-invariant, for both □+ and L o L are 
invariant while L, L are automorphic of certain degrees, so that averaging the equation 
□+ = QLL + FiL + F2L + F3 with respect to Redd produces an invariant Q and F3, and 
operators Fi, F2 automorphic of the appropriate degrees. Thus {LL + BiL + B2L + Bs)v G 
microlocally near F"'", where S2, -B3 map Hj to Hi for i = j — 1, j + 1, j respectively. 

Since W is strictly pseudo convex at all points not in A, it follows as in [K2] that 
on the complement of any neighborhood of A, the H^'^^ norm of v is majorized by 
C||n+v||^t+i(gvy) + C||v||^-jv, hence by C||£v||h= + C||v||j/-Ar + C||(5v||h«+2 with Q as in 
(1.3). This estimate is one derivative stronger than that which we seek. In particular, it 
now suffices to control the norm of v in an arbitrarily small neighborhood U of A, and 
to do so microlocally near F+. 

Fix a C°° cutoff function (f supported in a small neighborhood of A but identically 
equal to 1 in a smaller neighborhood, and fix an open set V disjoint from a neighborhood 
of A such that V</7 is supported in V. By Leibniz's rule and the pseudolocality of pseudo- 
differential operators, the norm of £,{Lpv — v) is majorized by C||f || +C||w||j7-iv 
for any N < 00. Thus by replacing v with ipv we may reduce matters to the case where v 
is supported in an arbitrarily small neighborhood W of A. Therefore it suffices to prove 
the existence of some W and an exponent s' < s such that (1.3) holds (with t replaced by 
s) for all V e Cq° supported in W. 

In a neighborhood of A in dW introduce coordinates {x,6,t) where 

Z2 = e"+^^ zi = e'^'^ie^'il - 0(2x)) - 1) 

with 2\x\ < r + 5 and \t\ < 5 for some small 5 > 0. In these coordinates A = {t = 0, \x\ < 
r/2}. Setting 

7(x, t) = 2 [e"^* - 1 + (j){2x) - i(j)'{2x)] / [l - (l){2x)] , 

the vector field L = + idg + 'jdt annihilates both zi and Z2- Hence it differs from what 
was previously denoted as L by multiplication on the left by a nonvanishing factor, which 
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may be verified to have the form b{x, t) exp(z^). For |a;| < r/2, (f){x) = and consequently 
^{x,t) = 2(e-** - 1). Therefore^ 

(2.2) L = + ide + ita{t)dt where < r/2, with Rea(O) ^ 0. 

The representation £, = LL + BiL + B2L + S3 in terms of the new L remains vahd with 
modified coefficients Bi that now preserve each Tij , once the operator formerly denoted by 
£ is multiplied by \b\~'^. 

There exists a unique C°° real-valued function /x, independent of 6, such that [L, L] = 
ifi{x,t)dt modulo the span of the real and imaginary parts of L. More precisely, since the 
coefiicients of do in [L, L] and in ReL vanish while the coefficient in ImL is nowhere zero, 

(2.3) [L, L] = iii{x, t)dt + w{x, t) Re L 

for unique real-valued, C°° coefficients |U, z/. The pseudo convexity of dW means that 
does not change sign. Replacing t by —t if necessary, we may assume that /x > 0. 

Fix an integer k. We identify functions of (x, t) e with elements of Hk via the corre- 
spondence u{x,t) I— > u{x,t)e'^^^; dg then becomes multiplication by ik. For the remainder 
of the paper we work in R^. Define 

V^{{x,t): \x\ < r/2 and t = 0}. 

By incorporating ik into the Bj we may further rewrite £, when restricted to Hk, as 

j!:o=ii + Bj+B2e + B3 

where ^ is a complex vector field in R^ which, for |a;| < r/2, takes the form 

e = + ita{t)dt. 

Here £ denotes the conjugate of £, each Bj is a pseudo differential operator of order < 
in R^, and ReQ:(0) ^ 0. Note that the commutator of the real and imaginary parts 
of I is not forced to vanish identically, because a is not real- valued. We have [£, i] = 
ili{x, t)dt + iv{x, t) Re£ with the same coefficients as in (2.3). 

Letting (^,r) be Fourier variables dual to (a;,t), define V = {(a;,t,^,T) : (x,t) G 
V and ^ = 0}. £0 is not elliptic at points of P, but is elliptic at most points in its comple- 
ment; r is the intersection of the characteristic variety of Cq with {(x, t, ^, r) : (x, t) e V}. 
Decompose F = r+ U F" where r+ = F fl {r > 0}. Thus the principal symbol of ijidt, 
namely — /xr, is nonpositive in a conic neighborhood of r+. 

^Boas and Straube [BS2] have shown the Q-Neumann problem to be globally C°° hypoelliptic whenever 
there exists a real vector field on the boundary that is transverse to the complex tangent space and has a 
certain favorable commutation property. If Rea(O) were to vanish then dt would be such a vector field. 
Thus nonvanishing of Rea(O) is for our purpose an essential feature of worm domains. 
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3. Two PseudodifFerential Manipulations. 

By an operator we will always mean a classical pseudodifferential operator, that is, one 
whose symbol admits a full asymptotic expansion in homogeneous terms of integral degrees. 
(7j{T) denotes the j-th order symbol of T (in the Kohn-Nirenberg calculus), always with 
respect to the fixed coordinate system (a;, i, ^, r). Henceforth we work under the convention 
that A, B, E denote operators whose orders are less than or equal to 0, 0, —1 respectively, 
whose meanings are permitted to change freely from one occurrence to the next, even 
within the same line. A denotes always an operator having the additional property that 
a"o(A)(x, t, ^, r) = for all (a;,t) G V. Any operator of type E may be regarded as one 
of type A. Two operators are said to agree microlocally in some conic open set if the full 
symbol of their difference vanishes identically there. 

Any operator B may be written in the form B = P{x) + E o i -\- A microlocally in a 
conic neighborhood of F"*", where /3 denotes both a function and the operator defined 
by multiplication by that function. Indeed, ao{B){x,0,0,T) depends only on (x,sgn(T)) 
and we define P{x) to be this quantity for r > 0. Then where r > and \x\ < r/2, 
(To(-B)(x, 0, ^, r) is divisible by ^ = — ■jcri(£)(a;, 0, ^, r). a"_i(-E')(a;, 0, ^, r) is then uniquely 
determined for such x,t by the equation cro(-B) = P{x) + 0'i{£) ■ a-i{E). Define E to be 
any operator of order —1 whose principal symbol satisfies this equation when restricted to 
{x,t) G V and to a conic neighborhood of F+. Then simply define A — B — (3 — Eo£. 

Writing Bi = Pi{x) + Ei o £ + Ai and similarly B2 = /32{x)+E2 £ + A2 , and expressing 
E££ = E££ plus an operator of order < 0, we obtain Cq = {I+E)££+{Pi+A)£+{P2+A)£+B. 
Composing both sides with a parametrix for I + E and modifying the definition of Cq 
to include this factor, we have Cq = ££ + (/3i + A)£ + {(32 + A)£ + B. Writing finally 
B ^ (3^{x) + Eo£ + A results in 

(3.1) Co = ££+ {(3i + A)£+ {(32 + A)£ + (/Ja + A) 

where the /9j are C°° functions depending only on x. □ 

We next reduce the question of a priori inequalities to L^, simultaneously for all 
s. Fix an operator Q of order that is elliptic in some conic neighborhood of F~, whose 
symbol vanishes identically in some conic neighborhood of F"*". Fix an exponent s > 0, for 
which we seek an a priori inequality for all v G C°° of the form 

(3.2) \\v\\h^ < C\\£ov\\h^ + C\\v\\h.' + C\\Qv\\hs+2 

for some s' < s. Having such an inequality for a sequence of exponents s tending to +00 
would imply Proposition 1 by the preceding discussion. In particular, the H^^^ norm of 
Qv is already under control by virtue of (1.1), while the norm of v is harmless because 
the Neumann operator is bounded on L^, and ||f Hji^^s' < + Cs^nW'^Wh-^ for any 

£ > and N <oo. 

Fix a C°°, strictly positive function m = m(^,r), homogeneous of degree 1 for large 
|(^,t)| and identically equal to (1 + r^)^/^ in a conic neighborhood of = 0}. Define A* 
to be the Fourier multiplier operator on with symbol m{^,T)^. 
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Substituting v = A~^u and g = A~*/, estimation of the norm of v, modulo a 
lower order norm, in terms of that of g is equivalent to estimation of the norm of u in 
terms of that of /, modulo a negative order norm of u. The equation Cqv = g becomes 
A~''CoA''u = f. Write A'^'liA^ = A~''£A^ o A~''£A^ and similarly for other terms, and 
note that A~^{j3j + A)A^ — jSj + A with the same function Pj. 

A=^* commutes with dx and with dt, so A~^{dx + iatdt)A^ = dx + A~*tA* o A~^aA^dt. 
Applying the Fourier transform gives immediately A~*[t, A*]9t = —s + E, microlocally in 
some conic neighborhood of F"'", modulo operators smoothing there of infinite order. Since 
A~^[q;, A^]9f is of order 0, 

A-W o A-'aA'dt = atdt + A-'[t, A']adt + t o A'' [a, A'] odt + E 
= ao {tdt - s) +tB + E, 

microlocally near F"*". Thus microlocally near F+j A~''£oA* = jCg becomes 

JC-s = 44 + iPi + A)es + {(32 + A)is + (/?3 + A) 

where Is As are first-order differential operators differing from respectively by terms of 
order zero, and taking the forms = dx ia{tdt — s), £s = dx — ia(tdt — s) for \x\ < r/2, 
where a depends only on t and the j3i only on x. 

To see that Ig does take the form claimed for |a;| < r/2, express I = dx + iatdt modulo 
terms '-y{x,t)dx and 'y{x,t)dt where 7 = for < r/2. Then A~^[^{x,t)dx,A'^] and 
A~^[7(x, t)dt, A^] are operators of the type A, since they have nonpositive orders and their 
symbols of order zero vanish identically for \x\ < r/2. □ 

4. A Two-Dimensional Problem And Preliminary Inequalities. 

The remainder of the paper consists of a self-contained analysis of a special class of 
pseudodifferential equations in a real two-dimensional region. We begin by describing 
the equations in question and fixing notation, which in some respects differs from that of 
preceding sections. 

Fix an interval / = [— r, r] C R. Denote by [x, t) G coordinates in a neighborhood 
U of V = I X {0}. The interval V corresponds to the degenerate annulus embedded in 
the boundary of the worm domain, and will be the focus of attention. The convention 
concerning the symbols A, S, E introduced at the outset of §3 remains in force. 

Consider a one parameter family of pseudodifferential operators of the form 

(4.1) jCs = LL+ (P^ix) + A)L + {(32{x) + A)L + {(3^{x) + A) 

where the fij are C°° functions. Suppose that L, L are first-order differential operators 
depending on the real parameter s, and that —L is the formal adjoint of L, modulo an 
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operator of order zero. Suppose that where |a;| < r, they take the special forms^ 

L^d^ + ta{x){tdt + s) + 0{t^)dt 
L = d^- ia{x){tdt + s) + 0{t^)dt. 

Here O(t^) denotes multiphcation by a smooth function divisible by on the region U. 
a and the coefficients Pj are assumed independent of s, but A and the terms 0{t^)dt are 
permitted to depend on s. 
Assume that 

(4.2) Re a{x) ^ for aU x G /, 

and that there exist smooth real-valued coefficients such that [L,L] = ifj,{x,t)dt + 
ii'{x, t) Re L, satisfying 

(4.3) > at every point of U. 

Because L* = —L modulo a term of order zero, L has the same real part as L. A 
change of variables of the form {x,t) ^ {x,h{x,t)), with h{x,0) = where < r, 
therefore reduces matters to the case where the real parts of both L and L are everywhere 
parallel to dx, and L = + ia{x, t){tdt + s) + 0{t^)dt on / x R, with a real-valued and 
nonvanishing. Rewrite a{x, t) = a{x) + 0{t), and incorporate the contribution of 0{t) into 
the various terms 0{t^)dt and A. (4.3) is invariant under diffeomorphism and hence fx 
cannot change sign, so the coefficient of t in the Taylor expansion of iJ.{x, t) about t = 
must vanish identically, for |a;| < r. This forces dxa{x) = there. Thus 

(4.4) L = dx + ia{tdt + s) + 0{t^)dt 

for \x\ < r, where a is a nonzero real constant. Moreover, may be expressed in t/ as a 
nonvanishing scalar multiple of L + L, modulo an operator of order zero. ^From now on 
we work in these new coordinates. 

Define F = {(a;,t,^, r) : (x,t) G X> and ^ = 0}. Decompose F = F+ U F~ where 
F+ = {r > 0} n F. Then by (4.3), in some conic neighborhood of F+ the principal symbol 
of [L, L] equals a nonpositive symbol, modulo terms in the span of the symbols of L, L 
and a term of order zero. 

The symbol || • ||, with no subscript, denotes the norm in L'^{U), while || • ||t denotes any 
fixed norm for the Sobolev space of functions having t derivatives in and supported 
in U. The goal of the remainder of the paper is the following a priori estimate. 

^No assumption is now made on the vanishing or nonvanishing of the coefficient of dt in L where \x\ > r, 
but the strict pseudoconvexity of W outside the exceptional annulus was used to reduce Proposition 1 to 
Proposition 2 below. 

9 



Proposition 2. Let {Cg} be a family of operators of the form (4-1) satisfying all of the 
hypotheses introduced above. Then there exist a discrete exceptional set S C [0, oo) such 
that for any s ^ S and any pseudodifferential operator Q of order whose principal symbol 
is nonzero in some conic neighborhood ofV~, there exist C < oo and a neighborhood W 
of T> such that for every C°° function u supported in W , 

\\u\\ + ||Lu|| + ||L«|| < C ■ (\\Csu\\ + + ilQwIli) . 

The key conclusions are that there is no loss of derivatives in estimating u in terms of 
CsU, and that this holds for a sequence of values of s tending to +oo. The assumption 
that u e C°° is essential. All hypotheses of §4 are satisfied by the family of operators 
Cs derived in §2 and §3. Proposition 2 thus implies the validity of (3.2), and hence of 
Proposition 1, which in turn implies our Theorem. 

Our first preliminary estimate is a standard one valid for all s e R. 

Lemma 1. For each exponent s and each Q there exists C < oo such that 

\\d^u\\ < C\\u\\ + C\\Csu\\ + C\\Qu\\i 

for every u e Cq°{U). 

Proof For (x^t) G cr2(£s)(x, t, ^, r) = if and only if (a;,t,^,T) G F. Therefore the 
characteristic variety of Cg in T*W is contained in an arbitrarily small conic neighborhood 
of F as 5 — > 0. Consequently there exists an operator Q of order zero such that firstly, T*W 
is contained in the union of the two regions where Q is elliptic and r > 0, and secondly, the 
symbol of Q is supported in the union of the two regions where Cs is elliptic, and where 
Q is elliptic. 

Since Cg is elliptic outside a small conic neighborhood of F, the norm of u is ma- 
jorized away from F by HvCs-uH + HwH-i, while in a conic neighborhood of F~ the norm 
of u is majorized by + 

Write (/, g) — Jjj fg dx dt. By Garding's inequality and the fact that i/i ■ ir < in the 
support of the symbol of Q, 

-Re{{LLu, u)) > c\\Luf +c\\Luf -C\\Lu\\ ■ \\u\\ - C\\Lu\\ ■ \\u\\ - C\\uf - C\\Qu\\l. 

The second condition imposed on Q ensures that 

\\Qu\\i < C\\Csu\\ + C\\Qu\\i + C||w||_i. 

Estimating {{Cs — LL)u, u) by Cauchy-Schwarz thus leads to 

\\Lu\\ + \\Lu\\ < C\\Csu\\ + C\\u\\ + C||Qw||i. 

But dx may be expressed as a linear combination of L and of L modulo an operator of 
order 0. □ 
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Lemma 2. There exists C < oo such that for every f e C^CR) and every £ > 0, 
Likewise 

\f{0)-f{-e)\<Ce'/'\\dJ\\r^.. 

The conclusions are invariant under translation, and the lemma will be invoked in that 
more general form. 

Proof. For each x G {£,2e), \f{x) — f{x — 3e)\ < j"^^^ \dxf{y)\dy and both conclusions 
follow from the triangle and Cauchy-Schwarz inequalities. □ 

To simplify notation define 

*B = \\jCsu\\ + \\u\\-i + 

Let 5 > be a small constant to be chosen in §6, and assume u to be supported in 

W C {{x,t) : \t\ < 5, \x\ <r + 5}. 

Applying Lemma 2 to the function x u{x,t) for each t and applying Lemma 1 gives 
the following estimate, under the hypotheses of Lemma 1. 

Lemma 3. 

\\dxU\\ + \\u\\ < C||w||l2(/x(-5,5)) +C^- 

5. Limiting Operators and Mellin Transform. 

Let a be a nonvanishing C°°, real- valued function. For e C define the ordinary 
differential operator 

-f^C = i^x +iCa{x)){dx -iCa{x)) + Pi{x){dx + iCa{x)) + P2{x){dx - iCa{x)) + Psix), 

acting on functions of a; e /. Only the case of constant a will be needed in this paper, but 
the general case arises in another problem and hence merits discussion. 

Definition 1. & is defined to be the set of all ( & C such that there exists a solution g of 
H(^g = on I, satisfying g{—r) = g{r) = 0. 

For any complex number w we write (w) = (1 + Iwp)^/^. 
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Lemma 4. & is a discrete subset of C, and for any compact subset K of [0, oo), the set of 
all C E & having real part in K is finite. For each s such that & fl (s + zR) = there exists 
C < oo such that for all ( G s + iH, for all f,(fi,ilj € C°°{I) satisfying H^f — (p + dxi/j, 
one has 

II/I|l^(/) + (C)-'I|5./IU^(/) 

< C(C)-^/'(l/(-r-)| + |/(r)|) + C(C>-^|b|U.(.) + C(C)-iVl|L^(/). 

Proof. Throughout this proof, aU norms without subscripts denote norms. The selfad- 
joint part of i?^, apphed to /, equals {dx-'ya{x)){dj:+'ya{x))f, modulo 0((7)||/|| + ||9a;/||), 
in the L^{I) norm. Therefore for s in any fixed compact subset of R and any = s + i^ E 
s + iR, for any / vanishing at both endpoints of /, 

-Re{Ha, f) > \\djf + 7' ^ \fW - 0((7)l|/f + 11/11 ■ \\dj\\). 
The coefficient a vanishes nowhere, while 

\{Ha, f)\ = \{^ + /)i < 11/11 • ii^'ii + wd.fw ■ \m. 

Combining the last two inequalities and invoking the Cauchy-Schwarz inequality and small 
constant - large constant trick, one obtains 

(5.2) 7'll/ll + l7|-||9./||<qiv'||+C^|7|-||V'll 

for all sufficiently large I7I, under the additional hypothesis that / vanishes at both end- 
points of /. 

There exists a unique solution of H(^(/)(^ = on /, satisfying (j)(^{—r) = 0, r) = 

1. Then (p(;{r) is an entire holomorphic function of C, and C G 6 -v^ 0(^(r) = 0. We have 
seen that C ^ ® provided that the imaginary part of ( is sufficiently large, when the real 
part stays in a bounded set. Thus 0^(r) is nonconstant, so has discrete zeros. ^ 

To prove (5.1) let ( = s + i'^ and / G C^i^) be given, and decompose f = g + h where 
H(^g = and h vanishes at the endpoints of /. The hypothesis ©fl (s + zR) = means that 
the Dirichlet nuUspace of Hq is {0}, so by elementary reasoning we conclude that for each 
7 there exists C < 00 such that \\h\\ + \\dxh\\ < C\\(p\\ + since H^^h = H(^f = ip-\-dj:ip. 
Moreover, since Hc^ depends continuously on C may be taken to be independent of Q 
in any compact subset of C\6. When I7I is sufficiently large, on the other hand, (5.2) 
implies + (C)"^ < C(C)"^||^|| + C(C)"^ HV'II- Thus the component /i of / satisfies 

(5.1). 



"^An alternative method of proof would be to combine (5.2) with general results from the perturbation 
theory of linear operators [Ka], utilizing again the holomorphic dependence of on C,. 
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We have H^^g = 0, so that clearly ||^|| and \\dxg\\ are majorized by C\f{r)\ + C|/(— r)|, 
uniformly for C, in any compact set disjoint from ©. Assuming henceforth that I7I is large, 
the equation gives the inequality 

\\dl9\\<C^'\\9\\+C\^\-\\d:.9\\. 
Integrating by parts as in the proof of (5.2) yields 

(5.3) \\d,gf + ^^\\gf < C\g{-r)d,g{-r)\ + C\g{r)dMr)\. 

To control the right hand side we use the bound 

\dM-r)\ + \dMr)\ < Cli^d^gW + C\M-'/'\\dlg\\. 
Indeed, setting v = dxg, for any r' e [r — |7|~"'^,r] 

\v{r) - v{r')\ <C [ \d,v\ < C\^\-'/^\\d,v\\L2. 

Jr' 



Then 



pr pr 

v{r)\<\^\ \v{r) -v{r')\dr' +\^\ \v{r')\dr' 

ir-|7|-i -^r— |7|-i 



and the desired bound follows by Cauchy-Schwarz. 

Putting this into (5.3), introducing a parameter A e R"*" and applying Cauchy-Schwarz 
yields 

i%r+\\d.gr 

< CX\g{-r)f + CX\g{r)\' + CX'']^] ■ \\d,gf + CX-'\^n\dlgf 

< CX\g{-r)\' + CX\g{r)\' + CX-'\^\ ■ Wd^gf + CX-'\^\-\^%r + I'lM") 

< CX\gi-r)\' + CX\gir)\' + CX-'\^\ ■ \\d^gf + CA-^^'^' 



Choose A to be a large constant times |7|. Then the last two terms on the right-hand side 
may be absorbed into the left, leaving 

+ ll^.^f < C|7| • |^(-r)|^ + C|7| • |^(r)|2. 

Since g = f at the endpoints of /, this is the desired inequality for g. Adding it to that 
for h concludes the proof. □ 



13 



Definition 2. 

S = {s E [0, oo) : there exists 7 e R such that s — | + ^7 e &}. 

Lemma 4 guarantees that S is discrete. 

Specialize now to the case where a{x) = a, the real constant in (4.4). Define 

Ls ^{dx + ia{tdt + s)) o {d^; - ia{tdt + s)) 

+ f3i{x){dx + ia{tdt + s)) + (32{x){d^ - ia{tdt + s)) + (3^{x). 

Expanding the last term in the expression = £s + (Lg — Cs) gives 

L^w = $ + 

where 

(5.4) s ^\ t) 

^ ' * = tdtAu + Au 

To reach (5.4) we may for instance express tdt o 0{t^)dt as {tdt)'^A + tdtA + A, since 
multiplication by t is an operator of the type A. Likewise t^t] = 9^] + is 
an operator of type A, because cr_i([^,t]) = cdT-ao{A) vanishes identically for {x,t) e T> 
since (Tq{A) itself vanishes there. 

The partial Mellin transform of / with respect to the t variable is defined to be 

/•oo 

/>,7)= / f{x,t)t-'n-Ht, 
Jo 

provided that the integral converges. If f{x, ■) G C°°[0, 00) has bounded support for each 
X, then the integral defining f{x,'j) converges absolutely whenever 7 has strictly positive 
imaginary part, and f{x,j) extends to a meromorphic function of 7 G C, whose only 
possible poles are at 7 = 0, —i, —2i, Clearly 

{tdtfrix, 7) = i-ff{x, 7) 

for all such /. Consequently 

{hsuy{x, 7) = Hs+iju{x, 7) for aU 7 G C\{0, -i, -2i, ...}. 
The Mellin inversion and Plancherel formulas read 

f{x,t)^cf f{x,^)f^d^, r \f{x,t)\H-'dt^c' f |/>,7)|'rf7- 

^0 ^R 

It follows directly from the definitions that {t^/'^f)^{x, 7) = f{x, 7 + f ) for all 7 G R. Thus 
the Plancherel identity may be rewritten as 

/•oo poo /• 

/ \f{x,t)\'dt^ \ty'f{x,t)\H-'dt^c' |/>,7+i)|2d7. 

^0 ^0 ^R 
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6. Proof of the Main Estimate. 

We may now estimate u in terms of CgU. To begin, 



/x[0,<5) JJixIl 



\u{x,t)\^ dx dt = c' II \u{x,^ + ^)\'^ d'y dx. 



Assume that s ^ S, and write C = s — ^+i^. With defined as in (5.4), Hi^u{x,^+^) = 
^{x, 7 + I) + dx'^{x, 7 + f )• Applying Lemma 4 on / yields for each 7 e R 

/ u{x,^+l)dx<C [ \${x,^+^)\^^)-Ux + C [ \^ixn+^)\^{l)-^dx 
(6.1) Ji Ji Ji 

+ C|«(-r,7+f)|' + C|«(r,7+i)|'. 

Lemma 5. Assume W C {{x,t) : \t\ < S, \x\ < r + S}. Then there exists C < 00 such 
that 



/ / \^x,^ + dx < C5^\\uf + C^^ 

J JlxR 



and 



1 1 \^{x,^+^)f{^)-'^d^dx < C5'^\\dxuf + C^\ 
J Jixh 

Granting the lemma, we conclude from (6.1) that 

\Mh{ix[o,s)) <CS^\\u\\'^ + CS^\\dxuf + C^^ + C j \u{r,t)\'^ dt + C j \u{-r,t)\'^ dt. 

But by Lemma 2 and the assumption that u{x, t) = for |a;| > r + 5, this last term is 
dominated by C^H^aj^lp. Thus 

\\Ahiix[o,5)) < C5^uf + CSWdM? + c^'- 

All the same reasoning applies on the region / x (—5,0], after the change of variables 
t ^ -t. Thus 

II«IIl2(7x(-5,5)) < cs^^WM + \\dM\) + c®. 

Combining this with Lemma 3 gives 

ll^ll + \\dM\ < CS^^WM + \\dxu\\) + C5S, 

so choosing 6 to be sufficiently small gives < C^, concluding the proof. □ 
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Proof of Lemma 5. The principal term in the double integral of the lemma for $ is of 



course the contribution of CgU: 



ff \{Csunx,^+^)\'{^)-u^dx 

J Jixh 



< \{Csur{x,j+^)\" djdx < C\\Csuf, 

J JlxR 

as desired. 

Any operator A of order < satisfying (To{A){x, t, ^, r) = for {x, t) E T> satisfies 

\\Auf < CS^Wuf + C\\u\W 

for all u supported in W, as 5 — > 0. A typical term of $ resulting from hgU — CsU is 
{tdtYAu. Its contribution to the first double integral in Lemma 5 is 

// \{{tdtfAunx,j+^)\'{j)-U^dx<c[[ \{Aur{x,j+^)f d^dx 

JJix'R. JJixR 

= C \Au{x,t)\'^dxdt 

J Jlx[0,S) 

< C6^\\uf + C\\uf_^. 
A typical constituent of is the term tdfAu. Its contribution is dominated by 

C [[ \{tdtAur{x,^+^)\^{^)-U^dx<C [[ \{Aur{x,j+^)\^ djdx 
JJixh JJixr 

and the remainder of the calculation is as above. □ 

Comments. The author can advance no reason why the method of reduction to the 
boundary should be essential to this analysis. Working directly on W might well result in 
a shorter proof. On the other hand, the analysis in §§4-6 applies with minor modification 
to a broader class of equations unconnected with the 9-Neumann problem. 

Some refinements of this analysis and related observations will appear in [Ch2] . 
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